Image processing apparatus having a buffer memory for image data storage

ABSTRACT

Two local buffers are provided between an image processing unit and an image compression and expansion unit for compression into a predetermined format. Write and read control units serve to alternately use the two local buffers. As a result, process flow starting from the image processing unit to generate compressed image data by the image compression and expansion unit requires no main memory, whereby high-speed image processing is allowed with low power consumption.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a division of and claims the benefit of priorityunder 35 U.S.C. §120 from U.S. Ser. No. 13/230,068, filed Sep. 12, 2011,which is a division of U.S. Ser. No. 10/811,840, filed Mar. 30, 2004(now abandoned), and claims the benefit of priority under 35 U.S.C. §119from Japanese Patent Application No. 2003-093075, filed Mar. 31, 2003,the entire contents of each of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus.

2. Description of the Background Art

In response to a recent trend towards higher pixel resolution of adigital camera, higher transmission speed and higher image processingspeed of captured image data have been required. FIGS. 7 and 8 eachschematically show a flow of image data captured by a conventionaldigital camera. An interlace sensor 2 is used as a CCD sensor for imagecapturing in FIG. 7, whereas a progressive sensor 18 is used in FIG. 8.With reference to FIGS. 7 and 8, image processing operation of aconventional digital camera will be discussed below.

With reference to FIG. 7, when a CCD sensor for image capturing is theinterlace sensor 2, the interlace sensor 2 first converts capturedinformation as light about a subject into a digital signal which is thenoutputted as image data. The outputted a subject into a digital signalwhich is then outputted as image data. The outputted digital signal issubjected to noise control by correlated dual sampling, A/D conversioninto a digital signal and the like at a signal processing unit 3, and isthereafter sent to a main processor 4. The main processor 4 comprises asensor processing unit (hereinafter referred to as “SPU”) 6 forperforming signal processing such as white balance control on thereceived image data. Next, the processed image data is stored in a mainmemory 5.

The interlace sensor 2 outputs the captured image of one frame in twofields, namely, odd and even fields. However, image processing using theSPU 6 and later units requires peripheral pixel information, and hence,information only about the odd field or about the even field is notresponsive to the image processing. In response, subsequent processsteps are put on hold until the interlace sensor 2 outputs the imagedata of two fields to reproduce the image data of one frame as originalimage data 16 on the main memory 5.

After the image data of one frame is reproduced as the original imagedata 16, a real-time processing unit (hereinafter referred to as “RPU”)7 performs signal processing such as interpolation, color spaceconversion and pseudo color suppression on the image data written to aline memory 8 from the main memory 5. The image data processed by theRPU 7 is thereafter sent to an image compression and expansion unit 13through line buffers A24 and B25 on the main memory 5. For compressioninto the JPEG (Joint Photographic Experts Group) format, the linebuffers A and B each normally have a capacity of 8 lines.

More specifically, after the processed image data is sent to the linebuffer A24, the RPU 7 continues to process subsequent image data readfrom the original image data 16, which processed data is then sent tothe line buffer B25. The image data sent in advance to the line bufferA24 is concurrently sent to the image compression and expansion unit 13and is subjected to processing therein. The RPU 7 continues to beoperative to send subsequent image data to the empty line buffer A24,concurrently with which the image data sent in advance to the linebuffer B25 is sent to the image compression and expansion unit 13. Thatis, the two line buffers A24 and B25 become alternately operative.

The image data read from the line buffers A24 and B25 are subjected tocompression into a predetermined format such as JPEG at the imagecompression and expansion unit 13, and are thereafter sent as compressedimage data 17 to the main memory 5. The compressed image data 17 issubjected to processing by an image display unit 19 and a signalconversion unit 20, whereby an image is displayed on a display device21, or the compressed image data 17 is stored in a memory card through amemory card interface. The image display unit 19 uses a display buffer29 as a working buffer memory.

Unlike the interlace sensor 2, the progressive sensor 18 as a CCD sensorfor image capturing outputs the captured image of one frame at a time,and hence, generation of the original image data 16 on the main memory 5is not required. Except for this, image processing using the progressivesensor 18 follows the same flow as the foregoing image processing usingthe interlace sensor 2.

As a result of higher pixel resolution, the CCD sensors 2 and 18 eachhave a growing number of pixels per horizontal line, reaching such alength that exceeds the horizontal line length of the line memory 8 ofthe RPU 7 in some cases. On the occurrence of such excess, regardless ofwhether the CCD sensor 2 or 18 is used, the image data outputted fromthe CCD sensor 2 or 18 cannot be directly processed by the RPU 7. Inresponse, even in the use of the progressive sensor 18, the originalimage data 16 is generated once on the main memory 5, and thereafter thesubsequent process steps are followed. More specifically, with referenceto FIG. 9, the original image data 16 is divided into left and rightregions to avoid excess of the pixels per horizontal line of theoriginal image data 16 over the horizontal line length of the linememory 8 of the RPU 7. The left region including regional data P₁through P_((n/2)) is processed first to generate a region 27 includingregional data Q₁ through Q_((n/2)). Next, the right region includingregional data P_((n/2)+1) through P_(n) is processed to generate aregion 28 including regional data Q_((n/2)+1) through Q_(n). Finally,two regional data R₁ and R₂ on the left and right are synthesized toform intermediate data 26 of one frame on the main memory 5 which isthen processed by the image compression and expansion unit 13.

As discussed, for compression of a subject image captured by a digitalcamera into a predetermined format such as JPEG to be displayed on adisplay device or stored in a memory card, image data transmissionshould be repeated between the main processor 4 and the main memory 5through a bus 14. More specifically, a digital camera using theinterlace sensor 2 requires five transmissions through the bus 14 asshown in FIG. 7. Even a digital camera using the progressive sensor 18requires three transmissions.

As discussed, the CCD sensors 2 and 18 have a horizontal line of pixelsreaching a length that exceeds the horizontal line length of the linememory 8 of the RPU 7, in which case the regional data R₁ and R₂ on theleft and right separately processed are synthesized into theintermediate data 26 which is then held in the main memory 5. As aresult, the main memory 5 should be larger in capacity than required inthe case in which the CCD sensors 2 and 18 contain pixels per horizontalline which do not reach the horizontal line length of the line memory 8of the RPU 7.

The foregoing data transmissions are handled by a DMA control unit 15which realizes high-speed DMA transmission through the bus 14 with nointervention of a CPU. However, when the CCD sensors 2 and 18 are a5-megapixel-class CCD sensor with 2560 pixels high by 1920 pixels wideeach storing 16 bits of information, for example, the image dataoutputted from the CCD sensor 2 or 18 contains information as much asabout 10 megabytes. In light of a recent trend towards still higherpixel resolution of the CCD sensors 2 and 18, repetition of transmissionof such high-volume data causes heavily places a burden on a bus band,thus leading to increase in power consumption, lower processing speed,and the like.

SUMMARY OF THE INVENTION

The present invention is intended for an image processing apparatus inwhich an image processing unit writes image data to a buffer memory, anda compression unit reads the image data from the buffer memory toperform compression thereon.

According to the present invention, the image processing apparatuscomprises: an image processing part for performing image processing oncaptured data of an image; and a storage unit provided outside the imageprocessing part. The image processing part includes: an image processingunit for performing a predetermined process on the captured data toobtain image data, and outputting the image data; a buffer memory forstoring the image data sent from the image processing unit; and acompression unit for compressing the image data read from the buffermemory.

Accordingly, high-speed processing of the image data is realized withoutusing the storage unit. Reduction in power consumption is allowed aswell.

According to one aspect of the present invention, the buffer memoryincludes two buffer memories. The image processing apparatus furthercomprises a control unit for controlling read and writing to and fromthese two buffer memories. The control unit is operative in such amanner that while the image processing unit writes image data to eitherone of the two buffer memories, the compression unit selectively readsimage data previously stored in the other one of the two buffermemories. As a result, the image processing unit and the compressionunit are both allowed to perform sequential image processing without theneed for waiting for completion of data transmission to the buffermemories.

It is therefore an object of the present invention to provide an imageprocessing apparatus realizing high-speed image processing with lowpower consumption.

These and other objects, features, aspects and advantages of the presentinvention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows the configuration of a digital camera usingan interlace sensor according to a first preferred embodiment of thepresent invention;

FIG. 2 schematically shows the configuration of a digital camera using aprogressive sensor according to the first preferred embodiment;

FIG. 3 schematically shows the configuration of a digital cameraaccording to a second preferred embodiment of the present invention;

FIG. 4 schematically shows the configuration of a digital cameraaccording to a third preferred embodiment of the present invention;

FIG. 5 schematically shows the configuration of a digital cameraaccording to a fourth preferred embodiment of the present invention;

FIG. 6 shows process flow of regional data according to the fourthpreferred embodiment;

FIG. 7 schematically shows the configuration of a conventional digitalcamera using an interlace sensor;

FIG. 8 schematically shows the configuration of a conventional digitalcamera using a progressive sensor; and

FIG. 9 shows conventional process flow of regional data.

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Preferred Embodiment

FIG. 1 schematically shows a digital camera comprising an imageprocessing apparatus 1 according to a first preferred embodiment of thepresent invention. In FIG. 1, arrows indicate flow of image datacaptured by the digital camera. With reference to FIG. 1, the imageprocessing apparatus 1 comprises a main processor 4 and a main memory 5.The image processing apparatus 1 is connected through a signalprocessing unit 3 to an interlace sensor 2.

The main processor 4 comprises an SPU 6, an RPU 7 including a linememory 8, an image compression and expansion unit 13, a local buffer A9(first buffer memory) and a local buffer B10 (second buffer memory)serving to transmit data between the RPU 7 and the image compression andexpansion unit 13, a write control unit 11 and a read control unit 12for selecting which one of the two local buffers A9 or B10 to use, a bus14 for data transmission, and a DMA control unit 15 for controlling datatransmission through the bus 14 between the main processor 4 and themain memory 5.

The main memory 5 serves to temporarily store original image data 16generated in the main processor 4 by data processing of odd and evenfields sent from the interlace sensor 2 through the signal processingunit 3, and compressed image data 17 sent from the image compression andexpansion unit 13.

Except for the two local buffers A9 and B10 provided between the RPU 7and the image compression and expansion unit 13, and operation of themain memory 5 as a line buffer, the digital camera of the firstpreferred embodiment has the same configuration as that of theconventional digital camera.

Though not specifically shown, the main processor 4 further comprises aCPU for various types of operations, an external memory card for storingcaptured image data, hardware serving as an interface for storage intothe external memory card, and a storage device for storing a softwareprogram.

The operation of the image processing apparatus 1 having the foregoingconfiguration will be discussed below.

In the main processor 4, the SPU 6 receives image data from theinterlace sensor 2 through the signal processing unit 3 to generate theoriginal image data 16 on the main memory 5. Next, the RPU 7 storesimage data in the line memory 8 read from the original image data 16 andperforms signal processing thereon such as interpolation, color spaceconversion and pseudo color suppression. These process steps are thesame as those followed by the conventional digital camera discussed withreference to FIG. 7. Unlike the conventional digital camera, the digitalcamera of the first preferred embodiment sends the processed image datato the local buffers A9 and B 10 in the main processor 4, not to theline buffers A24 and B25 in the main memory 5.

More specifically, the processed image data of 8 lines is sent to thelocal buffer A9. The RPU 7 thereafter continues to process subsequentimage data of 8 lines read from the original image data 16, and sendsthe processed image data to the local buffer B10. Concurrent withtransmission of the image data from the RPU 7 to the local buffer B10,the image data stored in the local buffer A9 is sent to the imagecompression and expansion unit 13 for compression into a predeterminedformat such as JPEG. The compressed image data is thereafter sent to themain memory 5. The RPU 7 still continues to process subsequent imagedata of 8 lines read from the original image data 16 and send theprocessed image data to the empty line buffer A9. The image data storedin the local buffer B10 is concurrently sent to the image compressionand expansion unit 13 for compression into a predetermined format suchas JPEG. The compressed image data is thereafter sent to the main memory5. That is, while the RPU 7 sends image data either to the local bufferA9 or to B10, image data stored either in the local buffer B10 or in A9is sent to the image compression and expansion unit 13 to be compressedtherein. These operations are repeatedly performed, whereby thecompressed image data 17 is generated on the main memory 5.

FIG. 2 shows the configuration of a digital camera using a progressivesensor 18 as a CCD sensor. Except for the progressive sensor 18, thedigital camera shown in FIG. 2 has the same configuration as that of thedigital camera of FIG. 1. In FIG. 2, constituent elements which areoperative in the same manner as those of FIG. 1 are designated by thesame reference numerals. The operation of the digital camera using theprogressive sensor 18 is the same as that of the digital camera usingthe interlace sensor 2, with exception that no original image data 16 isrequired, and data from the SPU 6 is directly sent to the RPU 7 to beprocessed therein.

The write and read control units 11 and 12 select which one of the twolocal buffers A9 or B10 to use by making connection to the RPU 7 and theimage compression and expansion unit 13.

The line buffers A9 and B10 are only required to be not less than imagedata processed at one time by the RPU 7 in horizontal line length. Morespecifically, as long as the local buffers A9 and B10 have a horizontalline not less than one horizontal line of pixels of the CCD sensors 2and 18, the local buffers A9 and B10 may be the same as or shorter thanin horizontal line length the line memory 8 of the RPU 7. By way ofexample, when the CCD sensors 2 and 18 are a 5-megapixel-class CCDsensor with 2560 horizontal pixels, and the line memory 8 of the RPU 7has a horizontal line length with 4096 pixels, the horizontal linelength of the local buffers A9 and B10 may contain 2600 pixels or 4096pixels, as long as it is not less than the length of 2560 pixels.

As discussed, the use of the local buffers A9 and B10 reduces the numberof DMA transmissions of image data between the main processor 4 and themain memory 5 under control by the DMA control unit 15 until thecompressed image data 17 is generated. That is, the conventional digitalcamera requires five transmissions in the use of the interlace sensor 2and three transmissions in the use of the progressive sensor 18 as shownin FIGS. 7 and 8, respectively, whereas with reference to FIGS. 1 and 2,the digital camera of the first preferred embodiment only requires threetransmissions and one transmission in the use of the interlace sensor 2and the progressive sensor 18, respectively. A burden placed on a busband is lightened accordingly, whereby high-speed image processing isrealized. The first preferred embodiment further advantageously servesto reduce the frequency of use of the main memory 5 generally operatingat a higher voltage than the main processor 4 as well as to reduce thenumber of operations of the DMA control unit 15, thereby reducing powerconsumption of the image processing apparatus 1 as a whole.Additionally, provision of two local buffers allows continuous processflow from the SPU 6 to the image compression and expansion unit 13,thereby advantageously realizing high-speed image processing.

Second Preferred Embodiment

Under the conditions that the progressive sensor 18 is used as a CCDsensor, and the progressive sensor 18 has a horizontal line of pixelsthat does not exceed in length the horizontal line of the local buffersA9 and B10, the first preferred embodiment allows real-time dataprocessing flow from the SPU 6 to the image compression and expansionunit 13 by way of the local buffers A9 and B10 without generating theoriginal image data 16 once on the main memory 5 as shown in FIG. 2.When the CCD sensors 2 and 18 have a horizontal line of pixels thatexceeds in length the horizontal line length of the local buffers A9 andB10, however, data sent from the RPU 7 cannot completely be stored inthe local buffers A9 and B10, whereby such sequential processing becomesimpossible.

In a second preferred embodiment of the present invention, it will bediscussed what process a digital camera follows to handle the case inwhich the CCD sensors 2 and 18 have a horizontal line of pixelsexceeding in length the horizontal line length of the local buffers A9and B10.

FIG. 3 schematically shows a digital camera comprising an imageprocessing apparatus 1 according to the second preferred embodiment. InFIG. 3, arrows indicate flow of image data captured by the digitalcamera. The image processing apparatus 1 of the second preferredembodiment has the same configuration as that of the first preferredembodiment, whereas the local buffers A9 and B10 are operative indifferent ways between the first and second preferred embodiments. InFIG. 3, constituent elements which are operative in the same manner asthose of FIGS. 1 and 2 are designated by the same reference numerals.

As an example, assuming that the CCD sensors 2 and 18 are a6-megapixel-class CCD sensor with 3072 pixels per horizontal line, thatthe line memory 8 of the RPU 7 contains 4096 pixels per horizontal line,and that the local buffers A9 and B10 each have 2600 pixels perhorizontal line responsive only to a 5-megapixel-class CCD with 2560pixels per horizontal line, the local buffers A9 and B10 are overflowedwith data sent from the RPU 7 which prevents process flow. In response,in the second preferred embodiment, the two local buffers A9 and B10together serve as one continuous buffer memory as shown in FIG. 3. As aresult, the horizontal line length of the local buffers A9 and B10becomes responsive to data up to 5200 pixels per horizontal line whichis twice the horizontal line length of each one of the local buffers A9and B10.

The use of the local buffers A9 and B10 as one continuous buffer memorydoes not allow the RPU 7 and the image compression and expansion unit 13to follow sequential process flow by means of the alternate use of thetwo local buffers A9 and B10 as discussed in the first preferredembodiment. In response, a control signal is transmitted between the RPU7 and the image compression and expansion unit 13.

More specifically, the RPU 7 first processes image data of one line readfrom the original image data 16, and sends the processed image data tothe buffer memory as a unity of the two local buffers A9 and B10. At thetime when data transmission is completed, the RPU 7 stops the operation.The image compression and expansion unit 13 serves to process the dataread from the buffer memory. After receipt of all the data in the buffermemory, the image compression and expansion unit 13 sends a controlsignal to the RPU 7 notifying completion of data readout. On receipt ofthis control signal, the RPU 7 again processes image data read from theoriginal image data 16, and sends the processed image data to the buffermemory.

The second preferred embodiment is applicable even in the use of theprogressive sensor 18 as a CCD sensor. More specifically, data from theSPU 6 is not directly sent to the RPU 7, but is sent to the main memory5 once to generate the original image data 16.

In FIG. 3, the two local buffers A9 and B10 are conceptually shown asone buffer memory in physical connection in a horizontal direction. Theactual use of the local buffers A9 and B10 is such that under control bya software program, for example, the RPU 7 sends data to the localbuffer A9, and overflow data from the local buffer A9 is sequentiallysent to the local buffer B10. This process flow is also followed inreading to the image compression and expansion unit 13. When the writeand read control units 11 and 12 are collectively controlled, the secondpreferred embodiment is feasible in the configurations shown in FIGS. 1and 2. That is, the configurations of FIGS. 1 and 2 can be subjected tothe process flows of the first and second preferred embodiments.

The second preferred embodiment has been described in the use of the twolocal buffers A9 and B10 as discussed in the first preferred embodiment.As the simplest way to use a buffer memory, one local buffer may beprovided in the image processing apparatus 1 that has the same capacityas that of the line memory 8 of the RPU 7. The second preferredembodiment is also feasible by the use of such a local buffer and theforegoing control signal.

As discussed, even when the CCD sensors 2 and 18 have high pixelresolution with a horizontal line of pixels exceeding in length thehorizontal line length of the two local buffers A9 and B10, theforegoing use of one buffer memory as a unity of the local buffers A9and B10 realizes image processing. Even when only one buffer memory isprepared which has a horizontal line length exceeding the length of onehorizontal line of pixels of the CCD sensors 2 and 18, the use of theforegoing control signal realizes image processing. Like the firstpreferred embodiment, data transmission between the RPU 7 and the imagecompression and expansion unit 13 does not require the main memory 5 andthe bus 14. A burden placed on a bus band is lightened accordingly,whereby high-speed image processing and reduction in power consumptionare realized.

Third Preferred Embodiment

In the first and second preferred embodiments, the local buffers A9 andB10 serve to transmit data from the RPU 7 to the image compression andexpansion unit 13. The local buffers A9 and B10 may be operative foranother processing performed in the main processor 4. In a thirdpreferred embodiment of the present invention, the local buffers A9 andB10 are operative for image display on a display device 21.

For visual recognition of a subject to be captured, a digital camera mayuse an electronic viewfinder or a liquid crystal display. With referenceto FIGS. 7 and 8, in the conventional way of such visual recognition,image data sent from the CCD sensor 2 or 18 passes through the signalprocessing unit 3 and the SPU 6, and is thereafter sent through the bus14 to the image display unit 19 for conversion. The converted image datais further sent through the signal conversion unit 20 to the displaydevice 21 such as an electronic viewfinder or a liquid crystal display,whereby an image is reproduced on the display device 21.

For the purpose of visual recognition of a subject to be captured,reproduction of the subject should be continuously made on the displaydevice 21, and should not be limited to a capturing period. Accordingly,data should be continuously sent to the image display unit 19, thusheavily placing a burden on the bus 14 serving for this datatransmission.

In response, in the third preferred embodiment, the local buffers A9 andB 10 serve to transmit data to the image display unit 19 to lighten aburden on a bus band.

FIG. 4 schematically shows a digital camera comprising an imageprocessing apparatus 1 according to the third preferred embodiment. InFIG. 4, arrows indicate flow of image data captured by the digitalcamera. Constituent elements which are operative in the same manner asthose of FIGS. 1, 2 and 3 are designated by the same reference numerals.

Image data sent from the CCD sensor 2 or 18 is subjected to theprocessing at the SPU 6. The processed image data is thereafter sent tothe RPU 7 after generation of the original image data 16 in the use ofthe interlace sensor 2, whereas in the use of the progressive sensor 18,it is directly sent to the RPU 7. The captured image data from the RPU 7is sent through the local buffers A9 and B10 to the image compressionand expansion unit 13 and to the image display unit 19. The capturedimage data is subjected to the processing at the image compression andexpansion unit 13 during image capturing, whereas in visual recognitionof a subject to be captured, it is subjected to the processing at theimage display unit 19. During visual recognition of a subject to becaptured, display on the display device 21 is required to be at such alevel that allows recognition of a subjected to be captured, and doesnot require data corresponding to millions of pixels sent from the CCDsensor 2 or 18. Accordingly, image data sent from the RPU 7 duringvisual recognition is skipped with 240 pixels high by 320 pixels wide,for example. The skipped image data is subjected to the sequentialprocess flow as discussed in the first preferred embodiment includingwriting from the RPU 7 and reading to the image display unit 19 by meansof alternate use of the two local buffers A9 and B10 under control bythe write and read control units 11 and 12.

Continuity between the RPU 7 and the local buffers A9 and B10, andcontinuity between the local buffers A9 and B10, and the imagecompression and expansion unit 13 and the image display unit 19, arecontrolled by purpose-built hardware and a purpose-built softwareprogram having a switching function of continuity such as the write andread control units 11 and 12.

The output from the RPU 7 is sent to the local buffers A9 and B10, andthe output from the local buffers A9 and B10 is selectively sent eitherto the image compression and expansion unit 13 or to the image displayunit 19. That is, the local buffers A9 and B10 are selectively availableeither to the image compression and expansion unit 13 or to the imagedisplay unit 19. More specifically, between the image compression andexpansion unit 13 and the image display unit 19, the local buffers A9and B10 are allocated to the one placing a heavier burden on the bus 14.

As discussed, data from the RPU 7 is directly sent through the localbuffers A9 and B10 to the image display unit 19 without datatransmission through the bus 14 for image reproduction as required inthe conventional digital camera. A burden placed on a bus band islightened accordingly, to thereby reduce power consumption. Thelightened burden on a bus band further advantageously leads to enhancedprocessing speed of the image processing apparatus 1 as a whole.

Fourth Preferred Embodiment

As discussed in the description of the background art with reference toFIG. 9, in response to the case in which the CCD sensors 2 and 18 have ahorizontal line of pixels exceeding in length the horizontal line lengthof the line memory 8 of the RPU 7, the original image data 16 is dividedinto left and right regions. These left and right regions are separatelyprocessed at the RPU 7 by means of the line memory 8 to generate theintermediate data 26 on the main memory 5. The intermediate data 26 issubsequently subjected to the processing at the image compression andexpansion unit 13 to generate the compressed image data 17. Such processflow can handle the CCD sensors 2 and 18 having a horizontal line ofpixels exceeding in length the horizontal line length of the line memory8 of the RPU 7, whereas a region for storing the high-volumeintermediate data 26 is required in the main memory 5 that containspixels corresponding in number to those of the CCD sensors 2 and 18. Inresponse, in a fourth preferred embodiment of the present invention, itwill be discussed how the original image data 16 is processed withoutrequiring such high-volume intermediate data.

FIG. 5 schematically shows a digital camera comprising an imageprocessing apparatus 1 according to the fourth preferred embodiment. InFIG. 5, arrows indicate flow of image data captured by the digitalcamera. Constituent elements which are operative in the same manner asthose of FIGS. 1, 2, 3 and 4 designated by the same reference numerals.FIG. 6 is an explanatory view sequentially showing generation of data onthe main memory 5. With reference to FIGS. 5 and 6, the operation of thedigital camera according to the fourth preferred embodiment will bediscussed below.

When the CCD sensors 2 and 18 have a horizontal line of pixels exceedingin length the horizontal line length of the line memory 8 of the RPU 7,and thus process flow cannot sequentially proceed from the SPU 6 to theRPU 7, the original image data 16 is generated once in the main memory 5regardless of whether the CCD sensor 2 or 18 is used. Next, the originalimage data 16 is divided and is then subjected to the processing at theRPU 7 by means of the line memory 8. With reference to FIG. 6, theoriginal image data 16 is divided into left and right regions in twocolumns. The left and right regions are each further horizontallydivided into regions of 8 lines. Hence, assuming that the CCD sensors 2and 18 contain 5000 pixels per horizontal line, for example, data to beactually processed contains half the number, namely, 2500 pixels perhorizontal line. As a result, the RPU 7 becomes operative even when theline memory 8 has a capacity of as small as 4096 pixels. Regional dataP₁ shown in FIG. 6 after being subjected to the processing at the RPU 7is written as data Q₁ to a line buffer A22 in the main memory 5. Unlikethe conventional way which subsequently processes regional data P₃,regional data P₂ is subsequently processed and is added as data Q₂ tothe line buffer A22 on the main memory 5. As a result, intermediate dataR₁ (23) of 8 lines is generated in the main memory 5 that has ahorizontal line length of pixels corresponding in number to those of thehorizontal line of the CCD sensors 2 and 18. The image compression andexpansion unit 13 handles data on a block unit basis of 8 pixels high by8 pixels wide, and hence, is capable of handling data R₁ to generatedata S₁. These process steps are sequentially performed until thecompressed image data 17 is generated.

As discussed, the RPU 7 processes the regional data including P₁, P₂, .. . , and the resultant data R₁, R₂, . . . are subjected to theprocessing at the image compression and expansion unit 13. Even when theCCD sensors 2 and 18 have a horizontal line of pixels exceeding inlength the horizontal line length of the line memory 8, process flowcontinues without waiting for generation of intermediate data of oneframe on the main memory 5 as required in the conventional digitalcamera. The fourth preferred embodiment further advantageously lowersworking buffer capacity in the main memory 5. Still further, provisionof two lines buffers A22 and B23 in the main memory 5, and alternate useof these buffers A22 and B23 as discussed with reference to the localbuffers A9 and B10 in the first and second preferred embodiments realizeimage processing with still higher speed.

Fifth Preferred Embodiment

The CCD sensors 2 and 18 may have a horizontal line of pixels exceedingin length the line memory 8 of the RPU 7 as a result of higher pixelresolution of the CCD sensors 2 and 18, in which case the fourthpreferred embodiment devises reading of image data from the originalimage data 16 to realize image processing. However, the fourth preferredembodiment requires a region for storing intermediate data in the mainmemory 5, thereby placing a heavier burden on the bus 14 and causinghigher power consumption than the first and second preferred embodimentsusing the local buffers A9 and B10. A fifth preferred embodiment of thepresent invention as a combination of the second and fourth preferredembodiments is responsive to these problems.

As a circuit configuration of a digital camera, the fifth preferredembodiment employs the one shown in FIG. 3 discussed in the secondpreferred embodiment. As to image processing, only the reading flow ofthe original image data 16 discussed in the fourth preferred embodimentwith reference to FIG. 6 is adopted.

More specifically, the original image data 16 generated on the mainmemory 5 after passing through the CCD sensor 2 or 18 and the SPU 6 isread as the regional data P₁ and P₂ in this order as shown in FIG. 6.The regional data P₁ and P₂ thereby read are subjected to the processingat the RPU 7 by means of the line memory 8. The processed data are sentto the local buffers A9 and B10, not to the line buffer A22 in the mainmemory 5. At this stage, the data R₁ (23) shown in FIG. 6 is generatedin the local buffers A9 and B 10. At the time when data transmission iscompleted, the RPU 7 stops reading of the original image data 16. Next,the image compression and expansion unit 13 reads the data from thelocal buffers A9 and B10 and performs compression thereon. Thecompressed data is thereafter sent as the compressed image data 17 tothe main memory 5. As this stage, the data S₁ shown in FIG. 6 isgenerated as the compressed image data 17 on the main memory 5. At thetime when data readout from the local buffers A9 and B 10 is completed,the image compression and expansion unit 13 sends a control signalnotifying completion of data readout. On receipt of this control signal,the RPU 7 continues to read the regional data P₃ and P₄ from theoriginal image data 16, and performs image processing thereon. Theprocessed data are sent to the local buffers A9 and B 10. These processsteps are repeatedly performed, whereby only the compressed image data17 is generated on the main memory 5 without generating the intermediatedata 26.

The fifth preferred embodiment has been discussed as a combination ofthe fourth and second preferred embodiments. As long as the CCD sensors2 and 18 have a horizontal line of pixels that does not exceed in lengththe horizontal line length of the local buffers A9 and B10, the firstpreferred embodiment may alternatively be combined with the fourthpreferred embodiment. Still alternatively, a combination of the thirdand fourth preferred embodiments will be applicable.

As discussed, the use of the local buffers A9 and B10 of the secondpreferred embodiment combined with the fourth preferred embodimenteliminates the need for generating intermediate data on the main memory5. Such combination further advantageously lightens a burden to beplaced on the bus 14 and reduces power consumption.

While the invention has been shown and described in detail, theforegoing description is in all aspects illustrative and notrestrictive. It is therefore understood that numerous modifications andvariations can be devised without departing from the scope of theinvention.

1. (canceled)
 2. An image processing apparatus for performing imageprocessing on captured data of an image of a desired subject,comprising: a storage unit provided outside an image processing part,said storage unit storing said captured data; and said image processingpart, including: an image processing unit for dividing said captureddata stored in said storage unit into two or more pieces of regionaldata in more than one column and in more than one row, performing apredetermined process on said two or more pieces of regional data forconversion into image data, and writing said image data to said storageunit, said two or more pieces of regional data being sequentially readfrom the first line and from the first column by means of DMAtransmission; and a compression unit for sequentially reading said twoor more pieces of regional data written to said storage unit by saidimage processing unit, and compressing said two or more regional data.3. An image processing apparatus for performing image processing oncaptured data of an image of a desired subject, comprising: a storageunit provided outside an image processing part, said storage unitstoring said captured data; and said image processing part, including: abuffer memory for storing image data; an image processing unit fordividing said captured data stored in said storage unit into two or morepieces of regional data in more than one column and in more than onerow, performing a predetermined process on said two or more pieces ofregional data for conversion into image data, and writing said imagedata to said buffer memory, said two or more pieces of regional databeing sequentially read from the first line and from the first column;and a compression unit for sequentially reading said two or more piecesof regional data written to said buffer memory by said image processingunit, and compressing said two or more regional data.